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Abstract 

This  paper  considers  resource  allocation  in  a  network 
with  mobile  agents  competing  for  computational  priority. 
We  formulate  this  problem  as  a  multi-agent  game  with  the 
players  being  agents  purchasing  service  from  a  common 
server.  We  show  that  there  exists  a  computable  Nash  equi¬ 
librium  when  agents  have  perfect  information  into  the  fu¬ 
ture.  We  simulate  a  network  of  hosts  and  agents  using  our 
strategy  to  show  that  our  resource-allocation  mechanism  ef¬ 
fectively  prioritizes  agents  according  to  their  endowments. 


1  Introduction 

Mobile-agent  systems  allow  user  programs  to  au¬ 
tonomously  relocate  from  one  host  to  another.  An  agent 
may  jump  to  one  site  to  filter  a  database,  jump  to  another 
site  to  access  a  camera,  and  to  a  third  to  process  the  re¬ 
sults  of  the  previous  two  hops  before  returning  the  results 
to  the  user.  For  each  of  the  three  hops,  there  may  be  alter¬ 
native  sites  to  access  compatible  resources  and  this  choice 
of  execution  location  subjects  hosts  to  greater  congestion 
volatility. 

Code  mobility  is  a  software  architectural  feature  that  has 
many  benefits  [12],  User  programs  can  reduce  the  effect 
of  network  latency  by  updating  their  execution  states  to  be 
closer  to  their  data.  Mobility  is  a  flexible  abstraction  that 
can  speed  software  development  and  deployment.  Addi¬ 
tionally,  mobility  provides  an  extra  layer  of  fault  tolerance. 

To  regulate  mobile  agents  and  provide  resource  owners 
with  greater  incentive  to  host  agents,  we  present  a  market- 
based  system  in  which  agents  bid  for  computational  priority 
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from  hosts.  We  derive  a  bidding  strategy  that,  given  perfect 
information,  computes  an  agent’s  bids  to  minimize  its  ex¬ 
ecution  time  of  a  sequence  of  tasks  under  a  fixed  budget 
constraint. 

The  added  dimension  of  agents’  choice  of  execution  lo¬ 
cation,  however,  exposes  hosts  to  additional  congestion  and 
volatility.  We  propose  that  agents  use  a  system  that  creates 
incentives  for  hosts  to  participate,  provides  agents  feedback 
about  the  costs  of  their  actions,  and  allows  agents  of  hetero¬ 
geneous  priorities  to  operate  simultaneously. 

We  construct  a  resource-allocation  policy  where  hosts 
take  bids  from  agents  for  prioritized  access  to  computa¬ 
tional  resources  (CPU  time).  The  priority  of  access  to  a 
resource  an  agent  receives  is  proportional  to  its  bid  relative 
to  the  sum  of  all  current  bids  at  the  host.  Hosts  collect  rev¬ 
enues  from  each  agent  at  a  rate  equal  to  the  agents’  bids. 

We  apply  this  policy  to  a  mobile-agent  system  with  sev¬ 
eral  different  types  of  resources  distributed  throughout  a 
network.  Each  agent  has  a  sequence  of  resources  to  con¬ 
sume  (an  itinerary)  and  an  endowment  of  electronic  cur¬ 
rency  to  be  used  to  purchase  resource  access  to  complete 
its  itinerary. 

We  formulate  the  hosts’  resource-allocation  problem  as 
a  game  with  the  players  being  agents  competing  for  a  re¬ 
source  from  a  common  server.  We  show  how  to  compute  the 
unique  positive  Nash  equilibrium  explicitly  under  perfect 
information  when  there  are  two  or  more  players.  Starting 
from  this  simple  mechanism  and  an  assumption  of  perfect 
knowledge,  we  develop  an  optimal  agent  bidding  strategy 
that  plans  an  agent’s  expenditure  over  multi-task  itineraries. 
Our  bidding  strategy  minimizes  execution  time  while  pre¬ 
serving  a  prespecified  budget  constraint.  We  complete  our 
work  by  presenting  a  simulation  of  mobile  agents  compet¬ 
ing  for  computational  access  in  a  network  of  heterogeneous 
hosts. 

The  paper  is  organized  as  follows.  In  Section  2,  we  de¬ 
scribe  the  system  model.  In  Section  3,  we  derive  the  optimal 
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bid  for  a  single  agent’s  first  job  given  load  statistics.  In  Sec¬ 
tion  4,  we  show  the  existence  of  a  Nash  equilibrium  that  can 
be  computed  by  the  server,  when  all  agents  submit  bid  func¬ 
tions  of  a  given  form.  In  Section  5,  we  show  that  the  Nash 
equilibrium  obtained  in  the  previous  section  is  unique.  In 
Section  6,  we  simulate  a  network  where  agents  submit  the 
optimal  policies  to  servers  that  allocate  resources  according 
to  the  resulting  Nash  equilibrium.  The  results  are  further 
discussed  in  Section  7,  and  some  related  work  is  described 
in  Section  8. 

2  System  Model 

The  system  model  follows  that  presented  in  [5],  We  con¬ 
sider  a  network  graph  where  agents  are  generated  at  some 
subset  of  nodes.  These  agents  are  given  a  task  of  completing 
a  set  of  jobs  of  different  types  in  a  given  sequence  by  pur¬ 
chasing  resources  from  service  providers  located  through¬ 
out  the  network.  An  agent  begins  with  an  endowment  of  /, 
dollars  to  spend  to  complete  its  task  and  wishes  to  minimize 
the  total  time  taken  to  complete  a  sequence  of  jobs  given  its 
budget  constraint.  We  assume  that  there  are  K  types  of  ser¬ 
vice  and  that  each  agent  only  needs  to  complete  a  job  of  a 
particular  type  at  most  once.  The  agent’s  task  can  be  rep¬ 
resented  as  the  sequence  {qk where  qlk  is  the  size  of 
the  fc-th  type  of  job  for  the  i-th  agent,  and  qk  =  0  implies 
that  the  agent’s  task  does  not  include  completing  a  job  of 
type  k.  We  assume  that  there  are  several  service  providers 
for  each  type  of  service,  and  the  capacity  of  the  provider 
chosen  by  the  i-th  agent  to  complete  its  job  of  type  k  is  ck. 
We  make  many  of  the  assumptions  for  the  sake  of  notational 
simplicity  and  the  following  analysis  can  easily  be  extended 
to  more  general  cases  of  multi-job  tasks. 

The  service  providers  wish  to  have  as  much  of  their  re¬ 
sources  utilized  as  possible,  and  thus  provide  their  entire 
capacity  at  no  cost  if  only  one  agent  is  requesting  service. 
If  more  than  one  agent  is  currently  requesting  service,  the 
capacity  is  partitioned  as  follows.  The  i-th  agent  receives 
service  at  rate  proportional  to  its  bid  relative  to  the  sum  of 
all  bids, 


(1) 


where  u\  is  the  amount  (in  dollars  per  second)  that  the  z-th 
agent  bids  for  service,  the  provider  receives  bids  totaling 
9k  from  the  set  of  agents,  Jk,  and  9~l  = 

Thus,  if  the  service  rate  is  constant,  the  time  taken  by  the 
i-th  agent  to  complete  its  job  of  type  k  is: 


- 


V) 


and  the  expenses  are: 


(2) 


9fc(«fc  +  V)  m 

ek  ~  ~i  • 

ck 

3  Single  Agent  Optimization 

The  problem  facing  the  z-th  agent  is  how  to  choose  its 
bids,  {ulk}k=1.  Computationally,  this  can  be  formulated  as 
an  optimization  problem  to  minimize  the  total  time  of  com¬ 
pleting  its  task,  Ti  =  Bit'Ll  such  that  the  budget  con¬ 
straint  is  preserved,  Ei  =  tkuk  <  li.  At  this  point, 

we  assume  that  the  agent  spends  at  a  constant  rate  u\  at  the 
provider  for  the  /z-th  type  of  job,  and  also  that  9kl  is  known, 
independent  from  and  remains  constant  throughout  the 
time  that  server  is  being  utilized.  In  this  section,  since  we 
are  dealing  with  the  i-th  agent  only,  we  drop  the  i  super¬ 
scripts  and  subscripts  in  all  our  variables  except  9kl.  We 
retain  the  notion  of  9k  l  to  differentiate  the  sum  of  bids  at 
the  host  submitted  by  competing  agents  from  9  k,  the  sum  of 
all  bids  including  the  i-th  agent’s,  to  simplify  our  derivation 
of  the  bidding  procedure. 

We  have  the  following  optimization  problem: 

K  K 

min  1\  s.t.  ek  <  I.  (4) 

fc= l  fc= l 

We  solve  this  problem  using  Lagrangian  methods,  and 
define  the  Lagrangian: 

£  =  fvfc  +  A(fyfc-l)  .  (5) 

fc=  X  \fc= 1  / 

Substituting  for  tk  and  ek  into  equation  (5)  and  taking 
partial  derivatives  with  respect  to  uk: 


p-  =  ^f  +  A«=0 

OUk  CkUk  Ck 


(6) 


Note  that  9] 7*  >  0  implies  A  >  0  for  all  but  the  trivial 
case  when  only  one  agent  bids.  Thus  we  have  the  following 
relationship  between  any  two  bids,  j  and  k: 


Incorporating  the  inequality  constraint,  we  get 


dC  ( v— <  qk(uk  +  9k  *) 

9A  - 


=  0. 


(8) 


Since  A  >  0,  it  follows  that  the  inequality  constraint 
must  be  satisfied  with  equality.  Substituting  for  {u  k}k=2  in 
terms  of  u\  using  the  relationship  in  equation  (7),  we  have. 
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{u\  =  max{0,  fi(01  *)}}iLi.  • 


(15) 


Solving  the  previous  equation  for  u  i,  we  get 


Ul 


T  _  V  _  9i  n-* 

J  Clpi 


9l  i  V  9fc 
ci  "i"  Cfc 


(10) 


which  yields  the  bid  for  the  first  job  for  the  *-th  agent  given 
the  loads  of  the  servers  for  the  jobs  in  its  sequence. 


4  Existence  of  Nash  Equilibrium  for  Multiple 
Agents  at  the  Same  Provider 

Consider  now,  without  loss  of  generality,  a  provider  with 
capacity  of  resource  type  1  and  N  agents  desiring  service 
are  currently  at  the  provider.  We  assume  that  the  agents 
have  some  estimate  of  future  loads,  9] 7\  so  that  they  are 
able  to  estimate  how  much  of  their  money  to  spend  for  this 
current  job.  An  estimate  can  be  obtained  by  augmenting  the 
system  with  “advertising”  agents  that  periodically  update 
and  make  available  the  values  of  total  bids  at  various  servers 
throughout  the  network.  A  simple  estimate  of  Of1  would 
be  the  means  of  the  loads  of  all  the  servers  of  type  k  jobs. 
Given  a  load  estimate  for  future  jobs,  the  optimal  bid  from 
equation  (10)  is: 


Intuitively,  a1  represents  the  estimate  of  the  money  avail¬ 
able  for  the  current  job,  and  if  that  is  less  than  zero,  the  agent 
cannot  afford  to  purchase  service  under  the  current  state  of 
the  network.  We  require  that  the  bids  be  non-negative.  We 
have  /3l  >  0,  9fl  >  0,  and  7*  >  0  with  equality  only  if  the 
agent  has  one  job.  If  a1  <  0,  /,  will  return  a  negative  value. 
Thus  the  agent  will  only  submit  a  bid  if  a*  >  0. 

At  the  server,  we  would  like  to  generate  a  set  of  bids  that 
forms  a  Nash  equilibrium  [2]  with  respect  to  the  policies  of 
the  N  agents: 


A  Nash  equilibrium  solution  is  a  set  of  bids  where  no 
agent  can  gain  an  advantage  by  unilaterally  changing  its 
bid.  One  possibility  of  reaching  the  Nash  equilibrium  is 
a  decentralized  algorithm  where  each  agent  makes  an  initial 
bid  and  then  updates  its  bid  at  preset  time  intervals,  t,  using 
the  iteration 


=  (16) 

where  tq(f)  and  6fx(t)  denote  the  i-th  agent’s  bid  and  sum 
of  competing  bids,  respectively,  at  time  t.  Unfortunately, 
this  algorithm  rarely  converges  to  a  Nash  equilibrium  and 
is  suboptimal  due  to  the  inconsistency  of  the  initial  guesses 
and  subsequent  iterations. 

Instead,  we  focus  on  a  centralized  method  to  obtain  op¬ 
timal  bids.  The  agents  submit  bid  functions  in  the  form  of 
equation  (11)  and  the  server  produces  the  optimal  bids  for 
each  agent.  To  formulate  this  new  method,  we  translate  each 
agent’s  bid  function  domain  from  Of1  to  a  single  common 
domain,  9\ .  Once  we  change  the  domain,  we  show  that  the 
space  over  ()\  is  continuous  under  most  conditions  and  that 
we  can  approximate  the  remaining  instances  with  our  exist¬ 
ing  framework. 

Using  u-space  has  a  deficiency  in  that  its  dimension  in¬ 
creases  with  the  number  of  agents.  To  reduce  our  search 
space,  we  iterate  over  a  common  domain  for  all  agents,  9\- 
space,  where  9\  :=  Of1  +  u\.  Modifying  the  policies  in 
equation  (15),  we  get  the  following  implicit  relations  be¬ 
tween  u\  and  9 1 : 


{u\  =  max{0,  fi(6i  -  .  (17) 

From  this,  we  can  obtain  an  explicit  function  gf9 1)  :  9\  — > 
u\.  Figure  1  illustrates  how  ft{9  —  u\)  shifts  as  9\  varies. 
Figure  2  demonstrates  the  shape  of  gi(9\).  Outside  the 
range  9\  £  (0,  a1  / (3l),  gi(9\)  takes  the  value  of  0.  We  now 
derive  <7(6*1 ). 

Substituting  6*  1  —  u\  for  9f  l  in  (11),  in  the  range,  9\  £ 
(0,  a1  / /?*),  we  have  the  following: 


u  1  = 


cF  -  ul) 

y/Si-ul 


(18) 


which  leads  to  a  quadratic  equation  in  u\.  Dropping  the  i 
superscript,  we  have: 


7  2u\  +  (a  —  (39i)2ui  —  (a  —  j39i)29i  =  0  (19) 

Taking  the  positive  root  of  the  equation  with  respect  to 
u  1,  we  have  u\  =  <7(6*1 )  where 
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Figure  1.  Behavior  of  max(0,  ft(0  -  u\))  for 

aV/94  =  4 


Figure  3.  Sample  plot  of  90 1)  versus  9± 

for  16  agents 


Figure  2.  Form  of  g0i) 


9(9  0 


(a  -  /36>i)2 
2'y2 


4726»i  \ 

(a  —  f36i)2  J 


(20) 


when  9 1  £  (0 ,  a1/ (3l)  and  u\  =  0  otherwise. 

We  see  that  g  is  continuous  at  9\  =0  and  9\  =  a/ (3. 
Thus,  g(9\)  is  a  continuous  function  of  9\.  We  also  note 
that  on  9\  £  (0,  a1  /  (3l): 


dg  _  2f3(a-f361) 

89!  ~  272 

-2/3(a~091f+2'l2(a-l391)~4f3~/2e1 
2j2^/  (a— /30i)2+4720i 


(21) 


Returning  to  the  question  of  the  optimal  bids  for 
N  agents,  we  seek  9\  and  to  satisfy  for  all 

agents  the  definition  9\  =  YliL iui  and  equation  (17). 
An  equivalent  problem  is  to  find  a  value  of  9 1  such 
that  Sil l  ui  -9i=  i  90 1)  -  9i  =:  =  0. 

We  know  that  dhi/ddi  |e1=o+  =  -1  +  J2iLi  1  >  0 
if  N  >  2  and  thus,  /ii  is  increasing  to  the  right 

of  zero  and  /ii(0+)  >  0.  We  also  know  that 

/ii(maxj{a7  / /?*})  =  —  maxj{a7//?}  <  0  for  the 

non-trivial  case  where  at  least  two  agents  have  a 1  >  0. 
Because  h\  is  the  sum  of  continuous  functions,  hi  is 
continuous  as  well  and  must  be  zero  for  some  value  of 
9\  £  (0,  ma Xi{az/P1}).  We  solve  for  this  value  by  using 
a  bisection  search  of  hi  in  the  given  range.  We  sketch  a 
sample  of  g0 1)  versus  9i  in  Figure  3. 

If  an  agent  has  only  one  job  left  to  complete,  it  can  be 
shown  that  ui  =  9i  for  9  £  (0 ,Ic/q)  is  its  optimal  pol¬ 
icy,  which  is  equivalent  to  having  7  =  0,  which  violates 
one  of  the  assumptions  made  earlier.  However,  by  using 
L’Hopital’s  rule,  we  see  that 


lim7^0+  g 


=  lim7^0+ 
=  lim7^0+ 


=  9i 


P1 7 


1  (<*-00i) 


if 


47V1+(“-^  1)^ 


1+ 


4-7^01 

(<*-(38 1 


and  when  9 1  =  0+,  we  have 


dg 

dOi 


01=0+ 


1 


2  (3a 


—2  (3a2  +  2'y2  a 


=  1.  (22) 


Thus,  if  we  require  agents  with  only  one  job  to  submit 
bid  functions  with  7  >  0,  we  allow  the  agents  to  approxi¬ 
mate  their  optimal  solutions  to  arbitrary  precision  and  still 
preserve  the  assumed  structure,  which  yields  an  equilibrium 
solution. 
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5  Uniqueness  of  Nash  Equilibrium  for  Multi¬ 
ple  Agents  at  the  Same  Provider 

Let  Oi  =  (0,  a1  / (3l)  be  indexed  such  that  0\  D  O2  D 
■  ■  ■  D  On  (i.e,  a1//?1  >  a2/ /32  >  •  •  •  >  aN /(3N)  where 
N  is  the  number  of  agents  at  a  server.  Let  us  define 

n 

K(0i)=J29i(0i)-0i.  (23) 

i= 1 

We  have  already  shown  that  hi(6i)  =  hi  {6x)  =  0  has  at 
least  one  solution  on  O  =  U  f—\Oi  =  (0,  //?*})  = 

Ox. 

Theorem  1  hx  {Ox)  =  0  has  only  one  solution  on  O. 

To  prove  Theorem  1,  we  must  first  prove  some  initial  lem¬ 
mas.  In  Appendix  9,  we  show  that  {d2 gx / d0\)  <  0  on  Oi. 
From  the  definition  of  h\l  in  equation  (23)  and  the  definition 
of  the  indices,  it  can  be  seen  that 


<  0  on  Oi  Vi  => 


dm 


=  1  Vi  => 


101=0+ 

<7i(  0)  =  0  Vi  => 


<  0  on  Or. 


d2hrl 
dO2 
dh% 

^1  0i=O+ 

/i?(  0)  =  0. 


=  n  —  1, 


(24) 

(25) 

(26) 


Also,  is  a  continuous  function  of  0 1 . 

Lemma  1  If  h{x)  is  a  twice  continuously  differentiable 
function  on  [r,  s],  ( d2h/dx 2)  <  0  on  (r,  s),  h{r)  >  0,  and 
h{s)  <  0,  then  there  exists  a  unique  point  Xq  £  (r,  s)  s.t. 
h{x  0)  =  0. 


Proof.  We  prove  this  lemma  by  contradiction.  The  Interme¬ 
diate  Value  Theorem  states  that  there  is  at  least  one  value 
Xq  £  (r,  s)  s.t.  h(x 0)  =  0.  Because  {d2h/d x2)  <  0  on 
(r,  s),  we  know  that  h  is  strictly  concave  on  (r,  s),  i.e., 


ah(x )  +  (1  —  a)h{y)  <  h{ax  +  (1  —  a)y)  (27) 

for  a  £  (0,1)  and  x,y  £  (r,  s).  Suppose  there  are  two 
points  that  satisfy  h{x)  =  0,  say  X\,X2  £  (r,  s)  where 
X\  <  x 2 .  Again,  using  the  Intermediate  Value  Theorem,  we 
can  show  that  3r0  £  (r,  xf)  C  (r,  s)  s.t.  h{rf)  >  0.  Then, 
we  have  ah{rf)  +  (1  —  a)h{ X2)  <  h(aro  +  (1  —  a)x2) 
which  implies  ah{r0)  <  h(aro  +  (1  —  a)x 2).  If 
a  =  (x2  —  X\/x2  —  ro)  £  (0, 1),  then  we  have 
ah(r<f)  <  h(x  1)  =  0,  which  is  a  contradiction  since  a  >  0. 
Thus,  there  can  be  at  most  one  point  where  h(x)  =  0.  <C> 

Proof  of  Theorem  1.  When  n  =  1,  {dh\/ d9\)\g1=0+  =  0, 
and  (d2h\/ dd\)  <  0  on  Ox,  which  implies  that  h\(9\)  <  0 


on  Ox-  When  n  =  2,  (dh2 /d6x)\e1=o+  =  1  and  h\ (0)  =  0, 
thus  /i?(0+)  >  0.  Also,  h\{a2 / (32)  =  h\(a2//32)  <  0 
and  (d2h\/dd\)  <  0  on  02-  Applying  Lemma  1,  we  get 
that  there  is  a  unique  point  6$  s.t.  h\(6f)  =  0  on  O2 ■  But, 
h\(Qx)  =  h\(6x)  <  0  on  0 1  IT  O2 thus  6*o  is  a  unique  point 
where  h'f(0o)  =  0  on  Ox- 

Uniqueness  can  be  shown  using  an  inductive  argument. 
Assume  that  there  is  a  unique  point  9 0  <  a1  / (3X  on  0\ 
where  h\(9o)  =  0.  Also  assume  (d2h\/ d9\)  <  0  on  O, 
and  h\{0\)  <  0  on  0 1  D  Of  Along  with  the  continuity  of 
h\,  the  previous  result  implies  the  following: 

r  >  0  <  e0 

h\(9x)  {  =0  0i  =  0o  (28) 

{  <  0  0i  >  0O 

Rewriting  equation  (23),  we  have  0-i+1  —  h\  +  gt+x- 
There  are  two  cases  to  consider: 

Case  1.  If  (cL+1//3l+1)  <  0o,  then  equation  (28)  is  satisfied 
for  /i(+1  because  3,+i(0i)  =  0  for0i  >  0o  >  (al+1  / (3l+1) 
and  t/i_|_i  (0i )  >  0  for  0i  <  (al+1  / (3l+1) .  Thus,  there  is  a 
unique  point  0o  where  /i(+1(0o)  =  0  on  Ox- 
Case  2.  If  0O  <  (ai+1/Pi+1)  <  (a1/^),  then 

/L1+1(cL+1//3l+1)  =  h\{al+1  / (3l+1)  <  Oby  equation  (28). 
We  also  know  ft|+1(0+)  >  0,  because  L(+1(0)  =  0  and 
(O/i(+1/i90i)|e1=o+  =  *  >  0.  Since  (92/i^+1/c) 92)  <  0  on 
Oi+\,  we  can  apply  Lemma  1  and  arrive  at  the  result  that 
there  is  a  unique  point  0o  on  Ot+i  where  /i(+1(0 0)  =  0. 
But  since  <j(j+i(0i)  =  0  for  0i  >  (al+1  / (3l+1) ,  we  have 
that  on  Oi  IT  Oi+ 1,  L(+1(0i)  =  h\  {9\ )  <  0.  Thus,  we  have 
a  unique  point  0o  where  /i(+1(0o)  =  0  on  Oi.  <(> 

6  Simulation  and  Results 

In  this  section,  we  define  an  algorithm  that  implements 
the  resource-allocation  policy  from  Section  4  and  describe 
a  simulation  of  the  policy. 

A  host  accepts  bid  functions  from  all  agents  present  any 
time  an  agent  arrives  to  or  departs  from  the  site.  Agents 
express  bids  through  three  coefficients  defined  in  equa¬ 
tions  (12-14).  The  host  takes  all  bids  to  form  the  bid- 
response  function,  g(6),  and  uses  a  bisection  search  to  find 
the  bidding  level  0  =  g(9).  Algorithm  1  sketches  this  oper¬ 
ation. 

We  base  our  simulation  on  the  Swarm  simulation  sys¬ 
tem  [13].  In  the  simulation,  there  are  100  hosts,  each  pro¬ 
vides  one  of  eight  computational  services,  and  the  capacity 
is  determined  from  a  truncated  Gaussian  random  variable. 
The  hosts  comprise  a  stochastically  generated  network  gen¬ 
erated  from  the  GT-ITM  package  [6],  In  GT-ITM,  a  net¬ 
work  is  built  from  a  hierarchical  system  of  transit  domains 
connecting  stub  domains.  The  user  specifies  the  number 
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Algorithm  1  Allocate  Resources  for  Host  k 

l:  while  true  do 

2:  t  :=  time  since  last  arrival/departure 

3:  for  all  agents  i  do 

4:  deduct  tgi{6)  from  agent  i’s  endowment 

5:  end  for 

6:  add  new  agent  or  remove  departing  agent 

7:  for  all  agents  i  do 

8:  query  agent  i  for  a,/3,  and  7  (equations  (12-14)) 

to  build  gi  (9) 

9:  end  for 

10:  search  for  9  =  J2Z=i9i(0)  in  (0,  maxi(a,//3i)) 

1 1 :  for  all  agents  i  do 
12:  v\  ■.=  ckgi{9)/e 

13:  end  for 

14:  end  while 


and  average  size  of  domains  in  nodes  (hosts)  as  well  as  the 
probability  that  nodes  are  connected  within  the  domain. 

Agents  are  created  at  a  Poisson  arrival  rate.  Their  start 
location  is  determined  uniformly  and  they  have  an  exponen¬ 
tially  distributed  number  of  jobs  in  their  itinerary.  Each  job 
size  is  chosen  from  an  exponential  distribution.  The  final 
parameter  describing  an  agent  is  its  endowment  size  rela¬ 
tive  to  the  sum  of  its  job  sizes.  We  generate  this  parameter 
from  a  truncated  Gaussian  random  variable  with  positive 
mean.  This  parameter  reflects  the  owning  user’s  preference 
that  the  task  completes  quickly. 

Once  an  agent  is  injected  into  the  network,  it  must  for¬ 
mulate  a  route.  It  chooses  a  route  incrementally,  choosing 
a  host  for  each  task  after  completing  the  previous  task.  For 
the  purpose  of  expenditure  planning,  however,  for  all  but 
the  next  immediate  host  choice,  agents  plan  to  visit  hosts  of 
average  capacity  ck,  and  average  congestion  Of-,  for  hosts 
offering  the  corresponding  service.  Agents  then  choose  the 
next  site  to  be  the  one  that  minimizes  the  sum  of  execu¬ 
tion  times,  assuming  no  change  in  bidding  level  9f 7\  and 
network  transfer  times  for  the  next  hop.  Thus,  our  routing 
algorithm  is  greedy  and  naive.  We  sketch  its  operation  in 
Algorithm  2. 


Algorithm  2  Choose  Next  Site  for  Agent  i 
1:  tmin  '■=  00;  nextHost  :=  0 
2:  for  all  hosts  k  offering  service  next  in  itinerary  do 
3:  tk  :=  [transf  erLatency :  k  from: 

currentHost]  +  Ckg(9^l)/(9^1  +  g(9 jj1)) 

4:  if  train  >  tfe  then 

5:  train  '■=  tk',  nextHost  :=  k 

6:  end  if 

7:  end  for 

8:  return  nextHost 


Figure  4.  A  screen  shot  of  the  simulator. 

Once  an  agent  jumps  to  a  site,  it  commits  to  finishing  the 
current  task  at  that  site.  To  complete  the  current  task,  the 
agent  submits  a  bidding  function  conforming  to  the  restric¬ 
tions  in  Section  5.  Whenever  an  agent  arrives  or  leaves  a 
host,  the  host  searches  for  an  equilibrium  congestion,  9. 

To  test  the  effectiveness  of  our  resource  allocation  pol¬ 
icy,  we  compare  the  endowment  of  agents  with  their  per¬ 
formance.  Once  the  network  has  reached  a  steady  state, 
we  designate  seven  percent  of  the  agents  injected  into 
the  system  as  test  agents.  Our  test  agents  have  identical 
itineraries  and  starting  hosts,  but  they  have  differing  endow¬ 
ment  sizes,  spanning  two  standard  deviations,  <r,  around  the 
mean  endowment,  //..  We  then  measure  the  performance  of 
agents  compared  with  the  performance  that  they  would  have 
achieved  in  a  network  with  zero  congestion.  We  compute 
this  idealized  measurement  by  computing  a  shortest  path 
through  the  network  where  the  distance  between  any  two 
hosts  is  the  sum  of  the  network  transfer  times  and  the  time 
an  agent  would  take  to  complete  a  job  at  the  host  without 
any  competition,  qk/ck- 

Figure  5  shows  a  plot  of  agent  endowment  versus  per¬ 
formance  relative  to  the  ideal.  The  figure  plots  the  mean 
performance  of  many  agents  at  various  endowment  levels, 
as  well  as  the  standard  deviations.  There  is  a  very  strong 
relationship  between  spending  and  the  speed  at  which  an 
agent  completes  its  itinerary.  We  show  a  y2  linear  fit  of  the 
performance  means  to  indicate  the  strength  of  the  relation¬ 
ship. 

7  Discussion  and  Future  Work 

We  have  presented  a  simple  resource-allocation  policy 
and  an  efficient  means  for  agents  to  plan  their  expenditure 
given  the  policy  and  perfect  information.  Because  agents  re¬ 
formulate  their  expenditure  plans  after  completing  every  job 
in  their  itinerary  and  the  bidding  strategies  preserve  agents’ 
budget  constraints,  theoretically  agents  will  always  com¬ 
plete  their  itineraries.  In  the  implementation  of  our  sim¬ 
ulator,  however,  we  use  discrete  time  units  and  round  up 
when  assessing  agents’  costs  to  the  host,  so  occasionally 
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Figure  5.  Endowment  versus  ideal  time  rel¬ 
ative  to  actual  time.  We  plot  the  observed 
mean  and  standard  deviations  for  each  en¬ 
dowment  level. 


agents  cannot  complete  their  itineraries.  This  occurrence  is 
less  frequent  with  finer  time-unit  granularity.  Further  agent 
failures  occur  when  agents  with  small  endowments  cannot 
complete  their  itineraries  because  higher  priority  congestion 
never  subsides. 

Our  planning  algorithms  rely  on  exactly  knowing  agents’ 
job  sizes.  This  assumption  is  reasonable  for  certain  appli¬ 
cation  domains.  However,  we  would  like  to  relax  this  re¬ 
quirement  in  our  future  work  by  exploring  other  strategies 
where  agents  had  probability  distributions  for  their  job  sizes 
instead  of  fixed  scalar  values. 

In  constructing  our  expenditure  planning  algorithm,  we 
assumed  that  agents  have  already  chosen  a  route  to  com¬ 
plete  their  itineraries.  In  the  implementation,  however,  we 
can  only  estimate  the  future  capacity  of  hosts  based  upon 
aggregate  host  statistics  and  we  choose  agents’  routes  in  a 
greedy  fashion.  This  technique  appears  to  work  well  for 
a  simple  network  topology  but  would  fall  short  in  a  more 
complex  network,  such  as  a  wireless  network  where  hosts 
move  about  their  environments.  Thus,  another  area  of  future 
research  is  to  investigate  better  ways  of  calculate  agents’ 
routes. 

Finally,  market  systems  require  that  agents  have  infor¬ 
mation  from  which  to  make  decisions.  Our  future  research 
will  attempt  to  evaluate  both  the  cost  of  distributing  infor¬ 
mation,  such  as  site  congestion,  as  well  as  the  cost  of  agents 
using  inaccurate  or  dated  information. 


methods  to  determine  the  price  of  a  certain  resource  in  a 
given  market  structure.  We  investigated  the  derivation  of  the 
equilibrium  pricing  scheme  in  a  single  seller  market  with  a 
number  of  buyers,  having  a  Cobb-Douglas  type  utility  func¬ 
tion  [4].  In  more  recent  work,  we  determine  prices  in  a  mar¬ 
ket  structure  in  which  the  servers  provide  a  price  curve  for 
the  resources  that  they  sell,  and  we  obtain  the  equilibrium 
price  as  the  result  of  a  utility  maximization  problem  [15], 

We  are  not  the  first  to  research  the  possibility  of  using 
market-based  control  for  mobile-agent  systems.  Telescript 
supported  a  system  where  agents  carried  permits  whose 
strength  diminished  over  times  and  as  they  moved  around 
the  network  [18].  The  weakening  of  permits  approximates 
currency  exchange.  The  Geneva  Messengers  mobile-agent 
system  includes  support  for  agents  to  buy  CPU  priority, 
memory,  and  network  access  [17],  Neither  system,  how¬ 
ever,  explores  how  agents  should  plan  in  market-based  en¬ 
vironments. 

POPCORN  is  a  market-based  system  where  users  sub¬ 
mit  Java  programs  to  a  centralized  server  that  contracts  the 
programs’  executions  out  to  servers  [16].  User  programs  in 
POPCORN,  however,  are  not  autonomously  mobile  and  do 
not  relocate  once  assigned  to  a  server. 

Modeling  telecommunication  network  problems  as  dy¬ 
namic  games  has  produced  Nash  equilibria  solutions  in 
many  settings  such  as  capacity  allocation  in  routing  [10], 
congestion  control  in  product  form  networks  [11],  flow  con¬ 
trol  in  Markovian  queuing  networks  [7],  and  combined  rout¬ 
ing  and  flow  control  [1].  Incorporating  electronic  market 
concepts  such  as  pricing  congestible  network  resources  has 
been  shown  to  encourage  efficiency  [14].  Network  resource 
allocation  where  users  were  charged  per  unit  time  has  also 
been  investigated  under  various  criteria  of  fairness  [8,  9], 

9  Conclusion 

We  presented  a  simple  means  of  resource  allocation  that 
prioritizes  agents  through  a  single  parameter,  endowment 
size.  To  utilize  the  mechanism,  we  derived  a  bidding  policy 
that  minimizes  an  agent’s  execution  time  for  an  itinerary, 
while  preserving  a  fixed  budget  constraint.  Furthermore,  we 
proved  that  the  use  of  our  optimal  bidding  strategy  results 
in  a  unique  computable  Nash  equilibrium.  We  simulated  a 
network  of  mobile  agents  and  their  hosts  using  our  bidding 
strategy  and  resource-allocation  policy  to  show  that  agents 
are  prioritized  by  their  endowments. 


8  Related  Work 

Our  earlier  work  in  this  area  adopted  a  currency  model 
of  resource  allocation,  in  which  agents  use  electronic  cash 
to  purchase  needed  resources  and  participate  in  an  elec¬ 
tronic  market  with  a  banking  system  [3].  There  are  several 


Appendix  Proof  of  Concavity  of  g *  {di ) 

For  the  variables  cL,/3%7*  defined  in  equations  (12-4) 
we  drop  the  superscripts  and  consider  the  bidding  function 
of  only  single  agent.  Let  w(x)  be  a  function  defined  as: 
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w(x)  =  — x 2  +  \/x4  —  bx3  +  bax2  (29) 

where  x  €  (0,  a)  and  b  =  (47 2 / (3). 

Then,  g\(9\)  =  i^w{a  —  (39\)  for  9\  G  0\  and 


d2g\/d9\  =  ((32 /2y2)(d2w/dx2).  (30) 

To  prove  the  concavity  of  g\  on  0 1,  it  suffices  to  show 
that  {d2w / dx2)  <  0  for  x  €  (0,  a).  We  have 


d2w/dx2  =  (2p(x)p"  (x)  —  p'{x)2  —  8p(x)z /4 p(x)%)  (31) 

where  p(x)  =  x4  —  bx3  +  bax2.  Since  p(x)  >  0  for  x  G 
(0,  a),  it  is  sufficient  to  show 


v(x)  =  2 \p{x)p"{x)  —p'( x)2  —  8 p(x)z  <  0.  (32) 

After  substituting  for  p(x)  and  simplifying,  we  get 

v{b ,  x)  =  —Aab2x3  +  12abx4  +  3 b2x4  +  8x6 
—12  bx5  —  8(x4  —  bx3  +  bax2)i 

We  note  that  v(0,  x)  =  0  Vx.  Taking  the  partial  deriva¬ 
tive  of  v(x)  w.r.t.  b,  we  get 


M  =  ~(a-jx)\6bx^+Ux2'y(x)\  -  2bax3 
7(0;)  =  sjx4  +  bx2  ( a  —  x)  —  x2 

hence  dv/db  is  negative  for  x  G  (0,  a)  and  b  >  0. 

v(b,  x)  <  0  for  all  b  >  0  for  x  G  (0,  a),  which  implies 
(d2w/dx2)  <  0  fora:  G  (0,a),  and  thus  (d2g\/d9\)  <  0 
on  0\. 
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